{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  TensorFlow入门（一）基本用法\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.7.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "print(tf.__version__)  # 查看 TensorFlow 版本"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow 是一个深度学习框架，大部分人使用 TensorFlow 都是为了实现一些神经网络模型。下面将按照下面几个步骤来感受一下 TensorFlow 的计算机制：\n",
    "- 学习 TensorFlow 的变量和常量\n",
    "- 学习 TensorFlow 的基本操作：加减乘除\n",
    "- 跑两个简单的例子来感受一下 TensorFlow 的计算机制"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.变量，常量\n",
    "### 1.1 变量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "v1: <tf.Variable 'w1:0' shape=() dtype=float32_ref>\n",
      "v2: <tf.Variable 'w1_1:0' shape=(1,) dtype=float32_ref>\n",
      "v3: <tf.Variable 'Variable:0' shape=(2, 1) dtype=float32_ref>\n"
     ]
    }
   ],
   "source": [
    "v1 = tf.Variable(0.1, dtype=tf.float32, name='w1', trainable=False)\n",
    "v2 = tf.Variable([0.1], dtype=tf.float32, name='w1')\n",
    "v3 = tf.Variable([[0.1], [0.3]], dtype=tf.float32)\n",
    "print('v1:', v1)\n",
    "print('v2:', v2)\n",
    "print('v3:', v3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到打印出来的结果中包含三个部分：\n",
    "- 第一部分是这个变量在命名空间中的一个名称，如 'v1:0'.因为神经网络中的参数一般很多，TensorFlow中使用了命名空间管理的方式让我们可以清晰地知道每个变量所在的位置和状态，这个在后面的学习中慢慢学习。\n",
    "- 第二部分 shape\n",
    "- 第三部分是变量的类型\n",
    "\n",
    "那我们给的变量值哪去了？这个就是 TensorFlow 比较奇葩的地方，我们需要初始化了以后，再启动 Session 才能得到它的值。说的很绕口，下面看看实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "init = tf.global_variables_initializer()  # 初始化操作\n",
    "\n",
    "sess = tf.Session()\n",
    "sess.run(init)   # 执行初始化操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们可以看到我们给的变量了,但是注意要通过 sess.run() 来看"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "v1 0.1\n",
      "v2 [0.1]\n",
      "v3 [[0.1]\n",
      " [0.3]]\n"
     ]
    }
   ],
   "source": [
    "print('v1', sess.run(v1))\n",
    "print('v2', sess.run(v2))\n",
    "print('v3', sess.run(v3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 常量\n",
    "理解了变量之后，其实常量就很容易理解了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c1 Tensor(\"Const:0\", shape=(1,), dtype=float32)\n",
      "c2 Tensor(\"Const_1:0\", shape=(2,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "c1 = tf.constant([1.0], dtype=tf.float32)\n",
    "c2 = tf.constant([1.0, 2.0], dtype=tf.float32)\n",
    "print('c1', c1)\n",
    "print('c2', c2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意：每次创建了新的变量、常量、操作以后都要执行初始化才能用。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c1 [1.]\n",
      "c2 [1. 2.]\n"
     ]
    }
   ],
   "source": [
    "sess = tf.Session()\n",
    "sess.run(init)   \n",
    "\n",
    "print('c1', sess.run(c1))\n",
    "print('c2', sess.run(c2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**trainable：** 细心的你可能已经发现，在上面的例子中，变量中有个 trainable 参数而常量中没有；这个很好理解，在神经网络训练的过程中，通过迭代不断更新参数，如果我们想要固定某一层的参数，那么我只需要把这层的参数设置为：\n",
    "```python \n",
    "trainable=False\n",
    "```\n",
    "\n",
    "上面我们简单的介绍了 TensorFlow 中的变量和常量，实际上在写网络结构的时候我们在定义网络结构的时候回更多的使用 tf.get_variable() 来创建变量。而且会结合 tf.variable_scope() 一块使用，这样能够方便地管理命名，同时还能够更好地实现权值共享。**这个现在看着可能会比较蒙圈，可以慢慢理解。**下面将介绍 tf.get_variable() 的用法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 使用 tf.get_variable() 创建变量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w1 <tf.Variable 'w1_2:0' shape=(3, 4) dtype=float32_ref>\n",
      "b1 <tf.Variable 'b1:0' shape=(3, 1) dtype=float32_ref>\n"
     ]
    }
   ],
   "source": [
    "w1 = tf.get_variable(name='w1', shape=[3,4], initializer=tf.truncated_normal_initializer())\n",
    "b1 = tf.get_variable('b1', [3,1])\n",
    "print('w1', w1)\n",
    "print('b1', b1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面我们使用 tf.get_variable 创建了两个变量。和刚才用 tf.Variable() 创建变量的方式有些不同，一般使用 tf.get_variable() 创建变量的时候我们会提供三个参数值：\n",
    "- name: TensorFlow 中的变量常量都是用这个 name 来区分的。为了便于区分，我把上面等号左边的 w1 称为变量名；把等号右边的name='w1'叫做对象名称，如上面的 **'w1_2:0'(自动回避了 tf.Variable() 创建的v1, v2 的命名)** 和 'b1:0',你可以理解为 'w1:0', 'b1:0' 在内存中对应两个地址。\n",
    "- shape: 这个变量的shape\n",
    "- initializer: 常见的初始化函数见：https://www.tensorflow.org/api_docs/python/tf/initializers\n",
    "\n",
    "在上面我们已经创建了一个名称 为 'w1' 的变量，**如果你再次创建一个变量，命名还是 'w1' 的话就有问题了。**\n",
    "\n",
    "看下面的报错：说 w1 变量已经存在了。这就相当于你在用 java 或者 C++ 写代码的时候写了：\n",
    "```java\n",
    "int a = 1;\n",
    "int a = 2; // 报错：a 已经存在\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Variable w1 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:\n\n  File \"<ipython-input-7-28eeb61e8bf0>\", line 1, in <module>\n    w1 = tf.get_variable(name='w1', shape=[3,4], initializer=tf.truncated_normal_initializer())\n  File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2963, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2903, in run_ast_nodes\n    if self.run_code(code, result):\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-90889f62983e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# 再创建一个变量，使用已经存在的 name，就会出错\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mw1_2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_variable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'w1'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m   1295\u001b[0m       \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1296\u001b[0m       \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1297\u001b[0;31m       constraint=constraint)\n\u001b[0m\u001b[1;32m   1298\u001b[0m get_variable_or_local_docstring = (\n\u001b[1;32m   1299\u001b[0m     \"\"\"%s\n",
      "\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m   1091\u001b[0m           \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1092\u001b[0m           \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1093\u001b[0;31m           constraint=constraint)\n\u001b[0m\u001b[1;32m   1094\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1095\u001b[0m   def _get_partitioned_variable(self,\n",
      "\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m    437\u001b[0m           \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    438\u001b[0m           \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 439\u001b[0;31m           constraint=constraint)\n\u001b[0m\u001b[1;32m    440\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    441\u001b[0m   def _get_partitioned_variable(\n",
      "\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_true_getter\u001b[0;34m(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m    406\u001b[0m           \u001b[0mtrainable\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrainable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    407\u001b[0m           \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 408\u001b[0;31m           use_resource=use_resource, constraint=constraint)\n\u001b[0m\u001b[1;32m    409\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    410\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mcustom_getter\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_get_single_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m    745\u001b[0m                          \u001b[0;34m\"reuse=tf.AUTO_REUSE in VarScope? \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    746\u001b[0m                          \"Originally defined at:\\n\\n%s\" % (\n\u001b[0;32m--> 747\u001b[0;31m                              name, \"\".join(traceback.format_list(tb))))\n\u001b[0m\u001b[1;32m    748\u001b[0m       \u001b[0mfound_var\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_vars\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    749\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_compatible_with\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfound_var\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_shape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Variable w1 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:\n\n  File \"<ipython-input-7-28eeb61e8bf0>\", line 1, in <module>\n    w1 = tf.get_variable(name='w1', shape=[3,4], initializer=tf.truncated_normal_initializer())\n  File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2963, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2903, in run_ast_nodes\n    if self.run_code(code, result):\n"
     ]
    }
   ],
   "source": [
    "# 再创建一个变量，使用已经存在的 name，就会出错\n",
    "w1_2 = tf.get_variable(name='w1', shape=[1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "细心的你可能发现在前面的例子中，使用 tf.Variable() 创建变量的时候我们给 v1 和 v2 的 name 是一样的，但是却没有报错；这里使用 tf.get_variable() 就不一样了。**实际上一般都是用 tf.variable_scope() 和 tf.get_variable() 来配合使用进行命名的。**在这里我们先不讲太难，这里只是先说明一下，后面再慢慢学习。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 一个简单的例子感受一下 TensorFlow 的用法\n",
    "用 tensorflow 实现计数器: 在循环中调用加法实现计数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "h_sum = 0.0\n",
      "vec =  [1. 2. 3. 4.]\n",
      "round 0: h_vec[0] = 1.0, h_sum =1.0\n",
      "round 1: h_vec[1] = 2.0, h_sum =3.0\n",
      "round 2: h_vec[2] = 3.0, h_sum =6.0\n",
      "round 3: h_vec[3] = 4.0, h_sum =10.0\n",
      "the mean is  2.5\n"
     ]
    }
   ],
   "source": [
    "h_sum = tf.Variable(0.0, dtype=tf.float32)\n",
    "h_vec = tf.constant([1.0, 2.0, 3.0, 4.0])\n",
    "\n",
    "# 把 h_vec 的每个元素加到 h_sum 中，然后再除以 10 来计算平均值\n",
    "# 待添加的数\n",
    "h_add = tf.placeholder(tf.float32)\n",
    "# 添加之后的值\n",
    "h_new = tf.add(h_sum, h_add)\n",
    "# 更新 h_new 的 op\n",
    "update = tf.assign(h_sum, h_new)\n",
    "\n",
    "\n",
    "sess.run(tf.global_variables_initializer())\n",
    "# 查看原始值\n",
    "print('h_sum =', sess.run(h_sum))\n",
    "print('vec = ', sess.run(h_vec))\n",
    "\n",
    "# 循环添加\n",
    "for i in range(4):\n",
    "    sess.run(update, feed_dict={h_add: sess.run(h_vec[i])})\n",
    "    print('round {}: h_vec[{}] = {}, h_sum ={}'.format(i, i, sess.run(h_vec[i]), sess.run(h_sum)))\n",
    "\n",
    "print('the mean is ', sess.run(h_sum / 4))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.tf.placeholder() 和 feed 的使用\n",
    "除了常量和变量之外，还经常用到的 tf.placeholder()。在神经网络中，我们一般都是每次喂一个 mini-batch 的数据，那么 tf.placeholder() 就相当于占了个坑，等着我们传一个 mini-batch 的数据来填它。\n",
    "\n",
    "tf.placeholder() 的作用就是创建占位符，后面再传一个 feed 参数来填它。<br/> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input1 Tensor(\"Placeholder_1:0\", dtype=float32)\n",
      "input2 Tensor(\"Placeholder_2:0\", dtype=float32)\n",
      "output Tensor(\"Mul:0\", dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "input1 = tf.placeholder(tf.float32)\n",
    "input2 = tf.placeholder(tf.float32)\n",
    "output = tf.multiply(input1, input2)\n",
    "\n",
    "print('input1', input1)\n",
    "print('input2', input2)\n",
    "print('output', output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([14.], dtype=float32)]\n",
      "[array([5.], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "# 通过 feed 来“填坑”\n",
    "_out = sess.run([output], feed_dict={input1:[7.0], input2:[2.0]})\n",
    "print(_out)\n",
    "_out = sess.run([output], feed_dict={input1:[5.0], input2:[1.0]})\n",
    "print(_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.一个简单的机器学习例子"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 例一：平面拟合\n",
    "通过本例可以看到机器学习的一个通用过程：1.准备数据  -> 2.构造模型（设置求解目标函数）  -> 3.求解模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X.shape= (100, 1)\n",
      "y.shape= (100, 1)\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "\n",
    "# 设置tensorflow对GPU的使用按需分配\n",
    "# 如果直接使用 sess = tf.Session() 的话会默认占用全部的 GPU 资源\n",
    "config  = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "sess = tf.Session(config=config)\n",
    "\n",
    "\n",
    "# 1.准备数据：使用 NumPy 生成假数据(phony data), 总共 100 个点.\n",
    "X = np.float32(np.random.rand(100)) # 随机输入\n",
    "X = np.sort(X)\n",
    "y = np.dot(0.200, X**2) + 0.300 + np.random.randn(100) * 0.01  # 加一点点噪声\n",
    "X = X.reshape([-1, 1])\n",
    "y = y.reshape([-1, 1])\n",
    "print('X.shape=', X.shape)\n",
    "print('y.shape=', y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 [[0.61981314]] [0.183312]\n",
      "20 [[0.2881131]] [0.27159673]\n",
      "40 [[0.22036755]] [0.29286206]\n",
      "60 [[0.20634001]] [0.2972653]\n",
      "80 [[0.20343544]] [0.29817703]\n",
      "100 [[0.20283401]] [0.29836583]\n",
      "120 [[0.20270947]] [0.29840493]\n",
      "140 [[0.2026837]] [0.298413]\n",
      "160 [[0.20267835]] [0.2984147]\n",
      "180 [[0.20267724]] [0.29841504]\n",
      "200 [[0.20267706]] [0.2984151]\n"
     ]
    }
   ],
   "source": [
    "# 2.构造一个线性模型\n",
    "b = tf.Variable(tf.zeros([1]))\n",
    "W = tf.Variable(tf.random_uniform([1,1], -1.0, 1.0))\n",
    "y_pre = tf.matmul(X**2, W) + b\n",
    "\n",
    "# 3.求解模型\n",
    "loss = tf.reduce_mean(tf.square(y_pre - y))         # 设置损失函数：误差的均方差\n",
    "optimizer = tf.train.GradientDescentOptimizer(0.5)   # 选择梯度下降的方法\n",
    "train = optimizer.minimize(loss)                     # 迭代的目标：最小化损失函数\n",
    "\n",
    "\n",
    "############################################################\n",
    "# 以下是用 tf 来解决上面的任务\n",
    "# 1.初始化变量：tf 的必备步骤，主要声明了变量，就必须初始化才能用\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)\n",
    "\n",
    "# 2.迭代，反复执行上面的最小化损失函数这一操作（train op）,拟合平面\n",
    "for step in range(0, 201):\n",
    "    sess.run(train)\n",
    "    if step % 20 == 0:\n",
    "        print(step, sess.run(W), sess.run(b))\n",
    "\n",
    "# 得到最佳拟合结果 W: [[0.100  0.200]], b: [0.300]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y_pre: [0.29845726 0.29847455 0.2985353  0.29858625 0.29862583 0.29885435\n",
      " 0.29893148 0.29920894 0.299272   0.2994445  0.30027884 0.30029494\n",
      " 0.30043533 0.3005978  0.30067915 0.30099547 0.30101013 0.3032228\n",
      " 0.30350995 0.30439574]\n",
      "y_true: [0.29525714 0.28868752 0.28505355 0.2989536  0.26959546 0.31568486\n",
      " 0.3057303  0.29026063 0.29732667 0.29826883 0.3082459  0.2898455\n",
      " 0.3077197  0.29528861 0.29756129 0.29852323 0.30031976 0.33165253\n",
      " 0.29205362 0.297387  ]\n"
     ]
    }
   ],
   "source": [
    "_y_pre = sess.run(y_pre)       \n",
    "print('y_pre:', _y_pre.reshape([-1])[:20])   # 预测值\n",
    "print('y_true:', y.reshape([-1])[:20]) # 真实值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，拟合的结果和真实值还是非常接近的。下面绘制一下最后的图看看吧："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8lNXd9/HPL2FT3AGtIohWvNWKdYlCnlpFRMQNaF2q1IIrtYLWervRW60Ftz69xVrLY0Xcq8UFxVAVBDXiEpTggoILCFZQUFmKK4Ekv+ePMwNhMslMklkz3/frNa9kZq5r5lxBv3Pmd53rHHN3RESkMBRluwEiIpI5Cn0RkQKi0BcRKSAKfRGRAqLQFxEpIAp9EZECotAXESkgCn0RkQKi0BcRKSBtst2AWJ07d/YePXpkuxkiInll7ty5K929S6Ltci70e/ToQWVlZbabISKSV8zs38lsp/KOiEgBUeiLiBQQhb6ISAFR6IuIFBCFvohIAVHoi4gUEIW+iEiaVVTAjTeGn9mWc+P0RURak4oKOOooWL8e2rWD556D0tLstUc9fRGRNCovD4FfUxN+lpdntz0KfRGRNOrbN/Twi4vDz759s9selXdERNKotDSUdMrLQ+Bns7QDCn0RkbQrLc1+2EepvCMiUkCSCn0zG2hmH5jZIjO7Ms7zZ5rZl2b2VuR2bp3nhpvZwshteCobLyLSWmRqWGfC8o6ZFQPjgaOBZcAcMytz9wUxmz7s7qNi9t0B+ANQAjgwN7LvmpS0XkSkFcjksM5kevqHAovcfbG7rwcmAYOTfP1jgBnuvjoS9DOAgc1rqohI65TJYZ3JhH5XYGmd+8sij8U6yczmmdljZtatifuKiLRKyZRt+vaF0XYTPyuakvZhnakavTMV+Ke7V5nZr4H7gH7J7mxmI4ARAN27d09Rk0REsivZsk1p5W2UVo/m7YPP5rLbhqR1pE8yPf1PgW517u8aeWwjd1/l7lWRuxOBg5PdN7L/BHcvcfeSLl0SLvEoIpIX4pVt6vX877sPLroIhgzhx7PvSPvQzmR6+nOAnma2OyGwTwOG1t3AzHZ29+WRu4OA9yK/TwduMLPtI/cHAKNb3GoRkTwQvRo32tPv1Gnznv+bV03mv64+Ozz4z39Cm/RfOpXwHdy92sxGEQK8GLjb3eeb2Rig0t3LgIvMbBBQDawGzozsu9rMxhI+OADGuPvqNByHiEjOib0at27P/8iqaex5zenQuzc8+SR06JCRNpm7Z+SNklVSUuKVlZXZboaISMpFa/y9q2bxdO0x1Pbcm46vvwDbbdfi1zazue5ekmg7XZErIpIhpaUw+7Y5TGt7Ar5bDzq+8mxKAr8pNPeOiEimzJvH/pcdA7t0hpdmQpcuVFRkdjI2hb6ISCa8/z707w8dO4ZCf9euWVlgReUdEZF0W7w4pHtRUUj23XcHsrPAinr6IiLptHRpCPyqqpDqe+218alOncJPs7DISiYWWFHoi4iky2efQb9+sHo1PP887LffxqcqKuDCC0MvHyBTAylV3hERSYcVK0Lgr1gB06bBwQdv9nR5OWzYsOl+dXVmyjsKfRGRVPv88xD4y5bBM8/EPTvbty+0bbvpfqbWz1V5R0Qklb78MtTw//3vEPiHHRZ3s9LS0LO///5wf9gwDdkUEckvK1eGwF+8GJ56Cg4/vNHNs7F2rkJfRCQVVq+Go4+GhQth6lQ48shstygu1fRFRJog7qIoa9aEwF+wAKZMCRdh5Sj19EVEkhT3Ctp918Ixx8A774TAP+aYbDezUerpi4gkKfYK2lenfQUDB8Jbb8HkyXDccdluYkLq6YuIJKnuoig7tP2a8544Ft6rhEcfhRNPzHbzkqLQFxFJUnRRlFemf8OIKcexzbuvwcMPw5Ah2W5a0hT6IiJNULr/t5T+/kR459WwxOFJJ2W7SU2imr6ISLK++gqOPRZmzYJ//ANOPTXbLWoy9fRFRJKxZk04afvGG6GHn4eBDwp9EZHEvvwSBgwI4/AnT4ZBg7LdomZT6IuINGb58nCx1ZIl4UrbAQOy3aIWUeiLiDQkugDKZ5+FydOOOCLbLWoxhb6ISDzRJQ5Xr4YZMzI/M1qaKPRFRGJ98EEI/O+/DytexSyAks8U+iIidb3zzqYJ08rLoVevrDYn1TROX0QKQtzZMWO98camJa1mzWp1gQ/q6YtIAYg7O2Zsib6iIlx4tf32YYM99shKW9NNPX0RafViZ8estwB5eXmYD79Ll9DDb6WBDwp9ESkA0dkxi4vjLEA+fXro4e+2Wwj8bt0Svl5SpaIcpfKOiLR60dkxy8tD4G8s7ZSVwSmnwD77hGGZXbokfK2kSkU5TKEvIgWh3iLkjzwCv/wlHHQQTJsWavlJiFcqyqfQV3lHRArP/ffD6adDnz6hh59k4EOCUlEeSCr0zWygmX1gZovM7MpGtjvJzNzMSiL3e5jZ92b2VuT291Q1XESkWe64A4YPhyOPDD38bbZp0u7RUtHYsflX2oEkyjtmVgyMB44GlgFzzKzM3RfEbLc18FvgtZiX+MjdD0hRe0VEmu/WW+Hii+H44+Gxx6BDh2a9TL1SUR5Jpqd/KLDI3Re7+3pgEjA4znZjgT8B61LYPhGRlnOHG24IgX/SSfD4480O/HyXTOh3BZbWub8s8thGZnYQ0M3dn4qz/+5m9qaZvWhmP21+U0VEmqG2Fi69FP7nf8KJ20mTQjG+QLV49I6ZFQHjgDPjPL0c6O7uq8zsYGCKmf3I3b+KeY0RwAiA7t27t7RJIiJBdTWcey7cdx9ceCH85S9QVNjjV5I5+k+Bulcr7Bp5LGprYD+g3Mw+BvoAZWZW4u5V7r4KwN3nAh8Be8W+gbtPcPcSdy/pksQ4WRGRhNatg5NPDoH/xz+Gen6BBz4k19OfA/Q0s90JYX8aMDT6pLuvBTpH75tZOXCpu1eaWRdgtbvXmNkeQE9gcQrbLyJS39q1MHgwvPgi3HYbjBqV7RbljISh7+7VZjYKmA4UA3e7+3wzGwNUuntZI7sfDowxsw1ALXC+u69ORcNFROJasSIsYD5/Pjz4IAwdmnifAmLunu02bKakpMQrKyuz3QwRyUcffRTWsF2xIixgPnBgtluUMWY2191LEm2naRhEpHV44w047rhw8vb556F372y3KCfprIaI5L9nnoHDD4f27eHllxX4jVDoi0h+mzgRTjwR9torTIG5997ZblFOU+iLSH5yh6uvhvPOCwugvPgi7LJLtluV81TTF5H8s359uOjqgQfgnHPg9tvDuraSkHr6IpJf1q4NJ2wfeADGjIE771TgN4F6+iKSP5YtC4H/3ntw771himRpEoW+iOSHefNC4H/1VRit07//Zk9XVMRZDlHqUeiLSO6bORN+/vOw4MnLL8P++2/2dL6vW5tJqumLSM6oqIAbbww/N7rvPjj2WOjRA2bPrhf4EH/dWolPPX0RyQn1euszndLnroNrrglPTJ4M224bd9/ourXRffNt3dpMUuiLSMY0Vnev21uvrdrAFr+9AConwrBhYYROIwufRNetVU0/MYW+iGREorp7tLferuprHuFUDqicBlddFYZlmiV8/XxetzaTVNMXkYy4//6wrklDdffSUpj18HLe/8ERHG0zYMIEGDs2qcCPJ+75AVFPX0TSr6IC7r47zJwA0KZNnLr7229TMvJEWLsapk4NJ29b8H4azROfevoiknbl5aGHD6HjftZZIYSjvfH3/zwVfvKTsIj5Sy+1KPCj76fRPPGppy8iaVH3pG3s6JphwyK98X7OBVW3sJdfyjd7H8RWz5WlZNI0jeZpmEJfRFIuXnkldnTNn8au59aqUZznd/I4P+ej0x7gsl22TMn7azRPwxT6IpJy8coro0fXCd+VKzn/iZPZ1l/kJhvN2PbXMXNAaqvNGs0Tn0JfRFKu0fLK/PkwaBDbfvopC//wD7z9L5nZVwGdKQp9EUm5BssrTz0Fp58OW24J5eX07NOH0VlsZyFS6ItIWmxWXnGHm2+Gyy+HAw6AJ5+Ebt2y2r5CpSGbIpJe338Pv/oVXHYZnHxyGJKpwM8ahb6INFnSV7suWwY//Sk8+CBcdx08/DB07JiRNkp8Ku+ISJMkfbXrK6/ASSfBd9+Fcs6gQRlvq9Snnr6INEnCq13dw0LlffvC1luHOfAV+DlDoS8iTRIdjllcHGc45rp1cO65cMEFMGAAzJkD++6bpZZKPCrviEiTxA7HhFDfH7DPUg6+4aQQ9FdfDddeC0XqV+Yahb6INFl0OGa0vt+7ahZn155C9Zbf0+aJJ2DIkGw3URqgj2ERAZo3/3z5C855627j2dqjWMP23DXidQV+jlNPX0SaN//8t99y7osj6OIP8SSDGdHhfqacuk1G2ivNp56+iDQ6IifuN4CFC6FPH7rM+Cef/Pp6Flz3OFOe30bz5+SBpHr6ZjYQuBUoBia6+00NbHcS8BhwiLtXRh4bDZwD1AAXufv0VDRcRFKnoQnS4n4DWP44nHlmeGDaNLoPGKD5c/JIwtA3s2JgPHA0sAyYY2Zl7r4gZrutgd8Cr9V5bF/gNOBHwC7ATDPby91rUncIItJSDU2QVvcbQG3VBoouvxJeHgeHHgqPPgrdu2ex1dIcyZR3DgUWuftid18PTAIGx9luLPAnYF2dxwYDk9y9yt2XAIsiryciOaa0NGbOezZ9A+hqnzGzth+9Xx4HI0fCrFkK/DyVTOh3BZbWub8s8thGZnYQ0M3dn2rqviKSPYlG7JSWwqO/nskbfgAH8CbD2z5ExS//Bu3bZ7ahkjItHr1jZkXAOODMFrzGCGAEQHf1HkQyIuGInZoa+OMfOe7W61jAPpzCo3xYuy97l2vBk3yWTE//U6DuPKi7Rh6L2hrYDyg3s4+BPkCZmZUksS8A7j7B3UvcvaRLly5NOwIRaZa69fqqqnAB7cYe/2efhU+EsWP58tjhHNHhdT4s3leLjLcCyYT+HKCnme1uZu0IJ2bLok+6+1p37+zuPdy9BzAbGBQZvVMGnGZm7c1sd6An8HrKj0JEmixary8qgtpamDkz5PyCvzwbFjqZMwfuvZcdn7qHqc93ZOzYJMfvS05LGPruXg2MAqYD7wGPuPt8MxtjZo1Onefu84FHgAXANGCkRu6IpE5zrqKNio7Y6d8/BL/VVnPVuqvY55KBsOOOIfSHD9+4bexJXslP5u7ZbsNmSkpKvLKyMtvNEMl5zbqKtoHXGdZvGXevG8pPeYnPTzyHnSb9NaxjK3nDzOa6e0mi7XRFrkieSjivfUTCETpfPMmCdj+md7s3WHjNA+xUNlGB34op9EXyVKPz2kdEvw1cdRUcfjhMmFDnyXXr4MILYcgQ2u7Zg/kPvMljHc5oVqlI8ocmXBPJUw1dRVtXeXkYmVNbG26jRkGvXlC6zXw4/XR45x343e+YPfhG+h3bvsWlIsl96umLtGJ9+26+jklNtfPVTf8PSkrg88/h6adh3DheeLV9UqUiyX8KfZE8FS3dXH11+BmvLFNaCuPHQ9u2sJN9QZkN5piykeHTYN48OPZYILlSkbQOKu+I5Kl4J3LjlWRGjICffjuNbtecyZbr/wN/ugUuumizrwDJlIqkdVDoi+SphqZD3sx338Hll7PP+PGw337w0IxQ1I8jugSitG4KfZE8lbB3PncunHEGvP8+XHxxGLfZoUMWWiq5RKEvksfi9s43bAgBP3Ys7LQTzJgRLrsVQaEv0rq89x4MGwaVlTB0KNx2G+ywQ7ZbJTlEo3dEWoPaWhg3Dg48EJYsCataPfigAl/qUU9fJN8tWRLWrJ01C048MVx2+4MfUFFRv94f7zEpLAp9kRbIaoi6w513wn//dxh+ec89YVZMs7iTsUFqJmiT/KbQF2mmVM1y2SwffwznnhvetF+/EPh1Vp1raDK2eOP61fsvLAp9kWZK9uKolKqthdtvhyuuALPw+4gRm8+1QMNj+GMfy+oHl2SFQl+kmZK6OCqVFi0KvfsXX4QBA0Jpp4E1pRsawx/72I03ZuGDS7JKoS/STBmbuqCmJgy9/P3vw6fLXXfBWWeFnn6C9sW2KfaxjH9wSdYp9EVaIO1TF8yfH3r3s2fD8cfDHXdA164peeloLf8vf4FVq1TTLxQKfZFcVFUFN9wQ6i/bbAMPPAC//GXC3n2yVMsvXLo4SyTXvPJKuMhqzBg49dRwle0ZZ6Qs8CH5pRal9VHoi+SKNWvg/PPhsMOoWv0tjwx/ioqR/4AuXVL+Vpo/v3CpvCOSbe7w0ENwySWwciWf/eJ3/PjJMaz5x1a0eyQ9NXfNn1+4FPoiGdDgBVALF8IFF8DMmXDIITBtGvdNO5A1j4XSS1VVWNe2tjb1tXfNn1+YVN4RSbO4yxpWVcEf/xgWNHn99bCmYUUFFesO5PXXw35FReFWU6Pau6SOQl8kzWJPmv77zmdD2F97LQwZEhY5ueACKl4vpm9fmDIlbGsWKj7t26v2Lqmj8o5ImkVPmu5YtZRb/Hf87J7JsOeeMG0aHHPMxu3Ky8P6J1G1tbDddqq9S2op9EXSJFrHP/L/VPHe8FvY+a6xFBc5XHN9mBmzffvNtu/bF9q2Dd8GYFPPXrV3SSWFvkgaROv4/aue4hS/mN18USjl3HIL9OgRd5/S0vAhcf/94f6wYQp7ST3V9EXSYN5jH/Lo9ydQVnsC1V7MpLOmwxNPbAz8iopwsW1Fxeb7lZaGsG9gHjWRFlNPXySV1q6F667jvL/eyjd04HL7M3e0v4hp57XbuEljUyBoegRJN/X0RVKhpiZMdbzXXnDzzRQN+xUfTv2Q7a+/lGnPt9ssuBubAkHTI0i6qacvEkeTVpN67rkwtnLePPjJT+Cpp6CkhBKg5IT6m/ftG4Zg1taGn3WHYWqqY0m3pELfzAYCtwLFwER3vynm+fOBkUAN8A0wwt0XmFkP4D3gg8ims939/NQ0XSQ9ki6xvPdeWMFq6lTYbTd45BE4+eSkJkaLbhK7qaZHkHRLGPpmVgyMB44GlgFzzKzM3RfU2ewhd/97ZPtBwDhgYOS5j9z9gNQ2WyR9Ei6D+Nln4cKqu+6CrbYKZ2Qvvhg6dEj69aurw5Q71dX1X19DNCWdkunpHwoscvfFAGY2CRgMbAx9d/+qzvYdAU9lI0UyqcESy9dfw5//DDffHK6iuvBCuOoq6Nw5qdeNlow6dVIJR7InmdDvCiytc38Z0Dt2IzMbCVwCtAP61XlqdzN7E/gKuMrdX2p+c0XSr16JpWQDjJ8Q5sr58kv4xS/g+uvhhz9M+jVjS0ZarUqyJWUnct19PDDezIYCVwHDgeVAd3dfZWYHA1PM7Ecx3wwwsxHACIDuGqAsOaC0FEr7OEyeDMN/H2bDPOKI0NM/5JAmv15syWjVKhg9OvXtFkkkmSGbnwLd6tzfNfJYQyYBQwDcvcrdV0V+nwt8BOwVu4O7T3D3Encv6ZKGBSNEmsQdZsyAQw+FU04JcyP861/wwgv1Aj/2IquGLrrSoiWSK5Lp6c8BeprZ7oSwPw0YWncDM+vp7gsjd48HFkYe7wKsdvcaM9sD6AksTlXjRVJu1qwwB/KsWWFEzr33hqUKi4vrbRqvZHPxxfFH/WhUjuSKhKHv7tVmNgqYThiyebe7zzezMUClu5cBo8ysP7ABWEMo7QAcDowxsw1ALXC+u69Ox4GItMjs2SHsZ86EnXeG226D886rNylaXbElm8mTGx/1o1E5kguSqum7+9PA0zGPXVPn9982sN9kYHJLGijZ0aSLk7KkqW2sqIgzmdncuXDNNfD007DjjjBuXFindostEr5H7Cifk06Cl17SqBzJbboiV+rJ1flf6gYwNK2NFRVhv+i0xXPumsezpX9gh1lTYIcd4KabwrqEHTtutk9j7xGvZNOrV+5/WEphU+hLPQkvTsqC2AAePnxTG9etCz34xtoYXaDkAN7kGsbwsw1T+ObVbfnkvDF0/9/fwjbbxN0n0d8htmSjEo7kOk24JvXk4kiT2ACGTedW3eGeezYfMRM7iuaEzrOZyom8yUEcyQtcyx/YrWYJe//jairm1w98yM2/g0hLqacv9eTiSJPY+vmwYeHxO+6oP53Bxm8FVc6ANs/zz1430Gvu8+y9zQ6U/dcYbq25kPK3tgsTnjXyTSYX/w4iLaXQl7hyrUzRUADfd1/9E6flz9cyoOpfXFl7A33Wv8bXH+4MN99M2xEjGLTVVnSJKRU11oPPtb+DSEuZe25Nk1NSUuKVlZXZbobkic1G1xxUBQ8+yHdj/5ctP36PJfTglrZXMPTZM+nTt0PD+ynUpRUws7nuXpJwO4W+5L3//CfUeW69FZYvhwMOYOGQy5hcfCpHHNVGoS4FIdnQV3lH8tcnn4SgnzABvvkGjj461Hv696enGVdmu30iOUihnyUqL7RARQXccgs8/ni4/4tfwKWXwoEHZrddInlAoZ8FuXrxU05bvx4eewz++ld47TXYdtuwROGoUaCZWUWSptDPgly8+ClnLV8e6vV33AErVsCee4Z5cc48M6xaJSJNotDPAi1+nYA7vPoqjB8fevcbNsCxx4aVqo45Bop0TaFIcyn0s0AX/TRg1Sp44AGYOBHmzw9TI1xwAYwcCT17Zrt1Iq2CQj9LcvGin6ycXK6tDYuTTJwYTsyuXw+9e4cROUOHbjYBmoi0nEJfgCycXP7007BAyV13wZIlsP32YUrjc86B/fdP4xuLFDaFvgDJnVxu8TeBDRvCvPUTJ4aftbXQr19YZPxnP4MOHVLzPiLSIIW+AIlPLrfom8CiRXD33WEqzBUrwspUV14JZ58NP/xh6t5HRBJS6OeBTPR8E51cbvIw0y+/ZPH/fYy2jz5It3+/EkbcHH88nHsuHHcctIn/n56Gs4qkl0I/x8VbfHvVqvR8ADR2cjneN4F6H0Zffw1PPgkPPYQ/+yx71NQwn30ZXXQTP7rhDM64omvCNmg4q0h6KfRzXN2e77p18JvfhMfbt29Z6aOp3x5ivwlA+DCiqoo32kzn9p8+ROdXy+D776F7d2b/5FJ+89JQ3vZeUGu0vRp+eHji99JwVpH0UujnuE6dQmWktjZcsxSdFLWqqvmlj+bWzTd+E6iq4pFfP8f4dY8x2Keww/o1fFfRGc46KwyzLC2F14qYfzhQHfatqUm+vbk4nFWktSjI0M+X0SEVFXDxxSEwzTYFPoQl/BoqfSQ6vqbWzSsq4OUZ33Ni+2fZe/5kKCvj1LVr+Q/bMtUG8VjboYyedhR9ftp24z6lpeGC2lGjwvu0b69SjUguKLjQz6fRIdFwrq0Nvf22bUOAFhXB3/4Wv93JHF/SdfPVq1n4l6dYcf2T/KZ2GlvxLdVbb0+bk38OJ5/M+x2PYtmr7bmyL/SJ05YRI6BXr+Z/wDb3wzlfPtRFsqHgQj+fRofEhnMyJ3GTOb5G6+Yffgj/+hdMnQovvUTPmhq2ZBce4FeUFf2MvpcfyRVXhR59H6DPEY0fQ/S1y8s3v59Icz+c8+lDXSQbCi7082l0SLyTp9HwbEiyx7exbv799zB9FjzzTLh9+GHYoFcvuOIK5u0xhNJRB1O1oYh27eCao5p2DM0N4eZ+OOfTh7pINhRc6Ofy6JB4ZYloOCcbngmPzx0++ABmzAghX14egr9Dh7DDRRfBCSfAbrsBsD8wc9/m/72aG8LN/XDOpw91kWwouNCH3BwdkijUmxKe9Y7vs8/CpGYzZ4bbsmXh8Z494bzzwrTFRxwBW2yR3Os1QXNDuLkfzrn8oS6SCwoy9FsiXScJE4V6k8JzxQqYNSsE/QsvhJ49hEnNjjoqrCXbvz/ssUfqDqABLQnh5n7Y5OKHukiuUOg3QbQ3XlUVRtCMHx9GqKRColBvMDzdYeFCePlleOmlcPvoo/DcVlvB4YeHqQ/69YMf/ziM9cwwhbBI7ijY0G9Oj728PAR+bW24jRoVznemItCS6RGXlkLp/t9CZSX8aTbMnh0O5PPPwwadO8Nhh4XLdg87DA46KIzzFBGJKMjQb+6Ikr59N10dC027yjQZ9XrE69bBu+/C3LkwZ064vfvupgb07BmWDzzssHDbe+9wFZeISAMKMvSbO6IkrVeZrlwJ77wD8+bxxbNv4W++SZcv5lNUE5nHYIcd4JBDYNAg6NMHevemYmHn8M1gPyjdJ0XtEJFWLanQN7OBwK1AMTDR3W+Kef58YCRQA3wDjHD3BZHnRgPnRJ67yN2np675zdOSYX0tusrUPZxkXbgw3BYsCD33d96B5cs3bcaOvMWBvN3meAZffxD/ddqBsPvum/XiE31b0VWpIhJPwtA3s2JgPHA0sAyYY2Zl0VCPeMjd/x7ZfhAwDhhoZvsCpwE/AnYBZprZXu5ek+LjaJKWDutr9MRkNNgXLdoU7tHfFy2Cb7/dtG2HDrDPPjBgQPgk6dWLv77Qi0v+vDM1NVDssGQpdH+4fjsb+7aiq1JFpCHJ9PQPBRa5+2IAM5sEDAY2hr67f1Vn+45AdGqwwcAkd68ClpjZosjrVaSg7S3S7BEltbWhFPPpp7B0KSxdyqcVn/DV24vpXrWIjssXwTffbNq+TZswNLJnTzjySNhzz/D7nnuGC6BiRtMcsjW0uzUEdnFxWGyqurp+eDf2bUVXpYpIQ5IJ/a7A0jr3lwG9Yzcys5HAJUA7oF+dfWfH7Jt4JY0USarEUV0Na9eG2+rVLHhpFYtmr2T/nT6nxxafh5ExdW9ffBH2qaMz7fiW3ZhV1JMDTj6CnQ/vuSncu3dvcJWoeOp+C/nkE7jzzs3DGzYdU0PfVnRVqog0JGUnct19PDDezIYCVwHDk93XzEYAIwC6d+/evAYsWwZ33BEW396wgeWfrOf9J9axa20VK+w71hz6Ldu3/Ra++y6UWL76KgR93XILsG/kBlDbth1FP9gJdtoJdtkFDjww/L7zztC1K++s2ZXLx+/G9De74BRBLZy/A9w+snmHUPdDavTocP+++zaFd6dO9cs2o0fXfx1dlSoiDUkm9D8FutW5v2vksYZMAm5vyr7uPgGYAFBSUuKxzydl+XK44YbQq27Xju1q2nJMTQeqaM/3vgXrlnaEPTuG0O7YEbbeGrbbDrYZytbAAAAGkElEQVTdliVrtuOG8dvyxYbtWUknVtKZlUU7cem12zL69/GHQFZUwFFnhGlrUqGhOnzd8G7RVAwiIiQX+nOAnma2OyGwTwOG1t3AzHq6+8LI3eOB6O9lwENmNo5wIrcn8HoqGl7PIYdQ8XJNveX8Noboo7BzAyE46Ua4pzYML4IwSKZDe+h7ZMNvd//9YRh9Xe3awbBhzWt+Q4EeG94q24hISyQMfXevNrNRwHTCkM273X2+mY0BKt29DBhlZv2BDcAaIqWdyHaPEE76VgMj0zVyJ15POdkSR90aeHExnH12CO+G9qmogLvv3rSSVdu2cM45je+TSDJ1eJVtRKSlkqrpu/vTwNMxj11T5/ffNrLv9cD1zW1gsuL1lEePbt6i34n2KS8P7wPhW8E558Dttze6S1yxJ5qTaYPKNiLSEq3mityWjlhpSpjGvldzSjoN1fAV6CKSTq0m9FtS+mjK1avRbZNZurAxmR5Lryt0RQRaUeg3V1OuXk3lla6ZHEuvK3RFJKrVhH4m1mJNZe88kydldYWuiES1mtDPxFqsqe6dZ6qGryt0RSSq1YR+bLB16gQ33pi4F92UHne+DpnM13aLSOqZe/MugE2XkpISr6ysbNa+0ZOVnTrBxRerhi0ihcPM5rp7SaLtijLRmEwpLQ1j81etql/qERGRVhb6UdFST3GxatgiInW1mpp+Xaphi4jE1ypDH3R1q4hIPK2yvCMiIvEp9EVECohCX0SkgCj0RUQKiEJfRKSAKPRFRAqIQl9EpIC0+tCvqAgTr1VUZLslIiLZ12ovzgItHiIiEqtV9/TjzbEvIlLIWnXoa+I1EZHNteryjiZeExHZXKsOfdDEayIidbXq8o6IiGxOoS8iUkAU+iIiBUShLyJSQBT6IiIFRKEvIlJAzN2z3YbNmNmXwL+T3LwzsDKNzclVOu7CouMuLM097t3cvUuijXIu9JvCzCrdvSTb7cg0HXdh0XEXlnQft8o7IiIFRKEvIlJA8j30J2S7AVmi4y4sOu7CktbjzuuavoiINE2+9/RFRKQJ8iL0zWygmX1gZovM7Mo4z7c3s4cjz79mZj0y38rUS+K4LzGzBWY2z8yeM7PdstHOVEt03HW2O8nM3MxaxQiPZI7bzE6N/JvPN7OHMt3GdEjiv/PuZvaCmb0Z+W/9uGy0M5XM7G4z+8LM3m3geTOzv0b+JvPM7KCUvbm75/QNKAY+AvYA2gFvA/vGbHMB8PfI76cBD2e73Rk67iOBLSO//6ZQjjuy3dbALGA2UJLtdmfo37sn8CawfeT+jtlud4aOewLwm8jv+wIfZ7vdKTjuw4GDgHcbeP444BnAgD7Aa6l673zo6R8KLHL3xe6+HpgEDI7ZZjBwX+T3x4CjzMwy2MZ0SHjc7v6Cu38XuTsb2DXDbUyHZP69AcYCfwLWZbJxaZTMcZ8HjHf3NQDu/kWG25gOyRy3A9tEft8W+CyD7UsLd58FrG5kk8HA/R7MBrYzs51T8d75EPpdgaV17i+LPBZ3G3evBtYCnTLSuvRJ5rjrOofQM8h3CY878lW3m7s/lcmGpVky/957AXuZ2StmNtvMBmasdemTzHFfC5xhZsuAp4ELM9O0rGrq//9Ja/UrZxUCMzsDKAGOyHZb0s3MioBxwJlZbko2tCGUePoSvtXNMrNe7v6frLYq/U4H7nX3m82sFHjAzPZz99psNywf5UNP/1OgW537u0Yei7uNmbUhfAVclZHWpU8yx42Z9Qf+Bxjk7lUZals6JTrurYH9gHIz+5hQ7yxrBSdzk/n3XgaUufsGd18CfEj4EMhnyRz3OcAjAO5eAXQgzE/TmiX1/39z5EPozwF6mtnuZtaOcKK2LGabMmB45PeTgec9cjYkjyU8bjM7ELiDEPitob4LCY7b3de6e2d37+HuPQjnMga5e2V2mpsyyfx3PoXQy8fMOhPKPYsz2cg0SOa4PwGOAjCzfQih/2VGW5l5ZcCwyCiePsBad1+eihfO+fKOu1eb2ShgOuFM/93uPt/MxgCV7l4G3EX4yreIcHLktOy1ODWSPO4/A1sBj0bOW3/i7oOy1ugUSPK4W50kj3s6MMDMFgA1wGXuntffaJM87v8G7jSz3xFO6p6Z7506M/sn4QO8c+RcxR+AtgDu/nfCuYvjgEXAd8BZKXvvPP/biYhIE+RDeUdERFJEoS8iUkAU+iIiBUShLyJSQBT6IiIFRKEvIlJAFPoiIgVEoS8iUkD+P61B8iplCi04AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(X, y, 'b.')\n",
    "plt.plot(X, _y_pre, 'r-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
